DOCUMENT RESUME 



ED 475 837 



TM 034 863 



AUTHOR 

TITLE 

PUB DATE 
NOTE 



PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Nix, Rebekah K.; Fraser, Barry J.; Ledbetter, Cynthia E. 
Evaluating an Integrated Science Learning Environment (ISLE) 
Using a New Form of the Constructivist Learning Environment 
Survey (CLES) . 

2003-04-00 

19p.; Paper presented at the Annual Meeting of the American 
Educational Research Association (Chicago, IL, April 21-25, 
2003) . 

Reports - Research (143) — Speeches/Meeting Papers (150) 
EDRS Price MFOl/PCOl Plus Postage. 

^Constructivism (Learning); Educational Environment; Factor 
Analysis; *High School Students; High Schools; ^Professional 
Development; Reliability; * Student Attitudes ; Validity 
^Constructivist Learning Environment Survey 



ABSTRACT 
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Learning Environment (ISLE) in public and private school classrooms. The 
CLES-CS, which measures students* perceptions on a 5-point frequency response 
scale, was administered to 1,079 high school students in 59 classes in north 
Texas to assess the degree to which the principles of constructivism were 
implemented in the class taught by the current teacher relative to classes 
taught by other teachers in the school. Principal components factor analysis 
with varimax rotation and Kaiser normalization confirmed the a priori 
structure of the CLES-CS. The factor structure, internal consistency 
reliability, discriminant validity, and the ability to distinguish between 
different classes and groups were supported for the comparative cases of the 
CLES-CS. When an analysis of variance was used to compare students* 
perceptions, of "this** and **other** classes, statistically significant 
differences were found for some CLES scales. Students whose teachers had 
attended the ISLE program perceived higher levels of Personal Relevance and 
Uncertainty of Science in their classrooms relative to the classrooms of 
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ABSTRACT. The purpose of this paper is to report on the validity and use of a new form of the existing 
Constructivist Learning Environment Survey (CLES). As part of a larger study, the comparative student 
version (CLES-CS) was developed to evaluate the impact of an innovative teacher development program 
(based on the Integrated Science Learning Environment model) in public/private school classrooms. We 
used a slightly-modified version of the CLES, originally developed by Taylor and Fraser (1991), whose 
design and orientation are informed by a psychological view of learning that focused on students as co- 
constructors of their own knowledge. Two response blocks for each of the same 30 items, comprising five 
scales, are presented in side-by-side columns. The instrument measures students’ perceptions on a 5-point 
frequency response scale of the extent to which certain psychosocial factors (Personal Relevance, 
Uncertainty of Science, Shared Control, Critical Voice, and Student Negotiation) are evident in THIS’ 
and ‘OTHER’ classroom learning environments. The CLES-CS was administered to 1079 students in 59 
classes in north Texas to assess the degree to which the principles of constructivism were implemented in 
the class taught by their current teacher relative to classes taught by other teachers in their school. 
Principal components factor analysis with varimax rotation and Kaiser normalization confirmed the a 
priori structure of the CLES-CS. The factor structure, internal consistency reliability, discriminant 
validity, and the ability to distinguish between different classes and groups were supported for the 
comparative cases (THIS and OTHER) of the CLES-CS. When an ANOVA was used to compare 
students’ perceptions of THIS and OTHER classes, statistically significant differences were found for 
some CLES scales. In particular, students whose teachers had attended the ISLE program (THIS) 
perceived higher levels of Personal Relevance and Uncertainty of Science in their classrooms relative to 
the classrooms of other teachers in the same schools (OTHER). 
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Evaluating an Integrated Science Learning Environment (ISLE) 
using a new form of the Constmctivist Learning Environment Survey (CLES) 

1. INTRODUCTION 

This paper specifically reports the validity and usefulness of a new comparative student 
form of the Constructivist Learning Environment Survey (CLES-CS). The overall research 
design involved evaluating the Integrated Science Learning Environment (ISLE) program in 
terms of promoting a more constructivist classroom learning environment, improving teachers’ 
attitudes toward information technology, and advancing teachers’ conceptual understanding. 

Data were collected throughout the study to investigate the general question of whether or not 
teachers’ participation in the ISLE program would lead to the teachers’ implementation of 
constructivist learning environments in their respective students’ school classrooms. Evidence 
derived from multiple sources was triangulated to ensure that the data were not contradictory, 
and therefore more likely to accurately describe the investigated item (Miles & Huberman, 

1984). Teacher and student perceptions of dimensions of the learning environment were used as 
dependent variables in the overall evaluation. 

1.1. Integrated Science Learning Environment (ISLE) 

Placing new content in personally-relevant contexts is the ultimate challenge of learning. 
Subjects traditionally perceived as distinct, such as the sciences, are particularly difficult to 
internalise and to apply in meaningful ways across variable situations. This ability to transfer 
knowledge and skills is critical in today’s changing society. In addition, decisions are no longer 
black or white, or right or wrong. Choices are typically based on selection from several 
possibilities. The ability to perceive the ‘bigger picture’ with innovative critical thinking and 
creative problem-solving skills is a new requirement for success. To keep up with today’s 
‘Nintendo Generation’, educators need a new perspective — and they need it now! 

The Integrated Science Learning Environment (ISLE) program modelled a constructivist 
paradigm to help teachers to learn and apply science content through creating a web page based 
on conceptual understanding represented in concept maps. Real-world applications of relevant 
information technologies were covertly employed to seamlessly bridge the gap between three 
traditionally separate milieus, namely, the university classroom, extended field trip, and 
public/private school classroom. The overarching research goal was to quantify the individual 
classroom learning environment through different views in terms of whether or not they had 
changed with the deliberate attempt at reform as presented in the ISLE program. 

1.2. Constructivist Learning Environment Survey (CLES) 

In response to the need to assess innovative classroom environments, like ISLE, the 
Constructivist Learning Environment Survey (CLES) was developed with a psychological view 
of learning that focused on students as co-constructors of their own knowledge (Taylor & Fraser, 
1991; Taylor, Dawson, & Fraser, 1995; Taylor, Fraser, & Fisher, 1997). The CLES was selected 
for use in this study because of its ability to characterise specific dimensions of the constructivist 
classroom. The five scales (Personal Relevance, Uncertainty of Science, Shared Control, Critical 
Voice, and Student Negotiation) enable a multidimensional assessment that provides the basis of 
the research design. Supporting this unique aim, a contemporary study by Allen and Fraser 
(2002) showed that the same questionnaire could be used to assess young students’ and their 
parents’ perceptions of actual and preferred classroom learning environment along the six 
dimensions of the What Is Happening In this Classroom questionnaire (WIHIC). The design and 
orientation of the instrument enabled a multi-dimensional assessment of the complex ISLE 
model. 
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Evaluating an Integrated Science Learning Environment (ISLE) 
using a new form of the Constmctivist Learning Environment Survey (CLES) 



Of specific interest with respect to this study, Dryden and Fraser (1998) used the CLES to 
assess the impact of a large-scale Urban Systemic Initiative (USI) aimed at changing high school 
science instmction toward a more constructivist approach. Unfortunately, the data reflected the 
state and district focus on increasing examination scores through professional development 
training with the direct delivery of program-specific information (i.e. content) rather than 
pedagogy in a general sense (i.e. context). However, the CLES was cross-validated with a large 
sample of approximately 1600 students in 120 grade 9-12 science classes in the Dallas 
metropolitan area. It is not only notable that this work was conducted in the north Texas area, but 
also that it validated use of the CLES with high school students in the same locale in which this 
study was conducted. Also relevant is the fact that the CLES was used to evaluate the 
constructivist-oriented reform of science education, as was the purpose of this study. 

In the overall study, three modified forms of the CLES were used to assess the perceived 
degree of constructivist teaching in the university by teachers and the school classrooms by both 
teachers and their students. The goal was to enable the classroom teachers to quantify the 
learning environment in terms of whether or not it changed with the deliberate attempt at reform 
as presented in the ISLE program through different views, as illustrated in Figure 1. 




Figure 1. Multilevel Assessment of ISLE Model Enabled by Three New Versions of the 

Constructivist Learning Environment Survey (CLES) 



Figure 1 shows how the different participants (university instructors, school teachers, and 
students) were able to evaluate two different learning environments (university/field trip and 
school classrooms) using three versions (adult, comparative teacher, and comparative student) of 
a single learning environment instrument (CLES). The adult form allowed the teachers to assess 
the degree of constructivist practice in the learning environment which they experienced as 
students in the imiversity setting. Then, the comparative teacher form allowed the same teachers 
to assess the degree of constructivist practice in the learning environments which they created as 
teachers in the school setting. This evaluation was supported by their respective students’ 
assessment of the degree of constructivist practice in the same school classroom learning 
environment. 
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Evaluating an Integrated Science Learning Environment (ISLE) 
using a new form of the Constmctivist Learning Environment Survey (CLES) 

Of primary methodological importance, the CLES provides a valid and reliable 
instrument for the assessment of how teachers’ and students’ perceptions of constructivist 
classroom learning environments change with first-hand experience (Fraser, 1998a). The 
established validity of the CLES was important when selecting it to answer the overarching 
research question of whether or not a teacher’s participation in the ISLE program would lead to 
the teachers’ implementation of constructivist learning environments in their respective students’ 
school classrooms. Consideration was also given to the cultural adaptability of the instmment 
(Lee & Taylor, 2001) for potential use in future cross-national and longitudinal studies based on 
the ISLE model. The CLES has been translated and validated for use in Korea (Kim, Fisher, & 
Fraser, 1999; Lee & Taylor, 2001) and Taiwan (Aldridge, Fraser, Taylor, & Chen, 2000). Of 
primary theoretical importance, the five scales of this particular learning environment instrument 
directly support the goals of educational reform in science described in the Adolescence and 
Young Adulthood/Science Standards (National Board for Professional Teaching Standards, 
2001). Table 1 matches the scales of the Constructivist Learning Environment Survey to the 
Science Learning Environment Standard stated as the primary goals for educational reform in the 
United States. 

Table 1. Constructivist Learning Environment Survey (CLES) Scales and Learning 

Environment Goals for Educational Reform in Science 



CLES Scale Science Learning Environment Standard Statement 



Personal “Teachers help students learn about and internalize the values inherent in the practice of 

Relevance science by relying on those values to shape the ethos of the learning community.” 



Uncertainty of “...they (the teachers) work diligently to establish a congenial and supportive learning 
Science environment where students feel safe to risk full participation, where unconventional 

theories are welcomed, and where students know that their conjectures and half-formed 
ideas will not be subject to ridicule.” 



Critical “...teachers recognize that the emotional response of some students to a lively, 

Voice argumentative, inquiry-based classroom might never to venture an opinion or idea, 

thereby avoiding the risk of public failure.” 



Shared “Accomplished science teachers deliberately foster settings in which students play active 

Control roles as science investigators in a mutually supportive learning community.” 



Student “They (the teachers) foster a sense of community by encouraging student interactions 

Negotiation that show concern for others, by dealing constructively with socially inappropriate 
behavior, and by appreciating and using humor.” 

Quoted from ‘Standard V: Learning Environments’ (National Board for Professional Teaching Standards, 
2001, p. 25) 



As evidenced by its widespread implementation, the Constructivist Learning 
Environment Survey (CLES) is a valuable tool to assist researchers and teachers in assessing the 
degree to which a classroom’s environment is consistent with a constructivist epistemology and 
to assist teachers in reflecting on their epistemological assumptions and reshaping their practice. 
Variations of the relatively short and highly appropriate instrument were made to make it suited 
to assessing both teachers’ and students’ viewpoints. 
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2. METHODS AND PROCEDURES 

Referring “to the social, physical, psychological, and pedagogical contexts in which 
learning occurs and which affect student achievement and attitudes” (Fraser, 1998b, p. 3), the 
field of learning environment research is broad in terms of both substance and methods. Based 
on the aims of the program and objectives of this study, a multilevel research design was selected 
to increase the understanding of the emergent model, giving special attention to the influence of 
the rapidly-developing field of information technology, within the classroom learning 
environment. To address the multi-faceted aspects of the new Integrated Science Learning 
Environment (ISLE), the research design was grounded in the naturalistic paradigm (Lincoln & 
Guba, 1985). The scales of the Constructivist Learning Environment Survey provided a critical 
scaffold for the development and use of new and revised evaluation resources for use with the 
ISLE program. In fact, the research methods employed were integrated into the overall design in 
an overt manner to model the evaluation and assessment of teaching and learning based on the 
constructivist paradigm. 

The combination of qualitative methods and quantitative measures (Fraser & Tobin, 

1991) in past studies has provided insight into the integrated milieu and evaluation of the near- 
and far-term effects of exposure to constructivist pedagogy. Other studies by Aldridge, Fraser, 
and Huang (1999) and Tobin and Fraser (1998) have successfully combined qualitative and 
quantitative research methods in studying the classroom learning environment at different ‘grain 
sizes’ to show how individual students and the teacher could be investigated also at the class 
level, school level, or system level. The comprehensive ISLE study was based primarily on 
quantitative data derived from learning environment dimensions in three modified forms of the 
CLES. Additional quantitative assessment, through attitude scales and concept map analysis, was 
supported by qualitative data derived from reflective field journals, interviews, and observations 
to suggest the impact of the emergent model. 

2.1. Comparative Student Form of the Constructivist Learning Environment Survey 

We developed a slightly modified version of the original CLES with two response blocks, 
each of the same 30 items (see Appendix) presented in side-by-side columns. The instrument 
measures students’ perceptions on a five-point frequency response scale of the extent to which 
certain psychosocial factors (Personal Relevance, Uncertainty of Science, Shared Control, 
Critical Voice, and Student Negotiation) are evident. The distinct feature of this version of the 
CLES is that it asks the student to provide perceptions not only of ‘THIS’ classroom 
environment (the student’s current class), but also of ‘OTHER’ classroom learning environments 
(other classes at the same school). 

The comparative student version of the Constructivist Learning Environment Survey 
(CLES-CS) was specifically designed for use with secondary school students. It incorporates 
some grammatical changes that were carefully made so as to maintain the validity of the original 
instrument in the new comparative format, which is illustrated below: 



I learn about the world outside of school. 


In OTHER classes... 




In THIS class. . . 








Almost ^ , , Some- 

Seldom . Often 

Never times 


Almost 

Always 


Almost c, , , 

Seldom 

Never 


Some- 

times 


Often 


Almost 

Always 
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The left, shaded area begins with ‘In OTHER classes. . while the right, clear area 
begins with ‘In THIS class.. The 60-item CLES-CS contains six statements in five scales 
about practices that could take place in a class or program. Table 2 lists the name and provides a 
description, along with a sample item, of each scale. 



Table 2. Scale Name, Scale Description, and Sample Items for the Constructivist Learning 

Environment Survey — Comparative Student (CLES-CS) Form 



Scale Name 


Scale Description 


Sample Item 


Personal Relevance 


Relevance of learning to students’ 
lives 


1 learn about the world outside of 
school. 


Uncertainty of Science 


Provisional status of scientific 
knowledge 


1 learn that science has changed 
over time. 


Critical Voice 


Legitimacy of expressing a critical 
opinion 


It's OK for me to ask the teacher 
‘why do 1 have to learn this?’ 


Shared Control 


Participation in planning, conducting 
and assessing of learning 


1 help the teacher to plan what I’m 
going to learn. 


Student Negotiation 


Involvement with other students in 
assessing viability of new ideas 


1 ask other students to explain their 
thoughts. 



2.2. Student Sample and Administration of the CLES-CS 

The CLES-CS was administered to 1079 students in 59 classes in north Texas 
represented by 12 teachers. The sample consisted of a diverse range of age, level, ability, and 
other demographic characteristics. In general, this assemblage represented nine independent 
districts, including eight different public schools and two private, parochial schools. With respect 
to individual classes, the number of students ranged from 5 to 30 per period, while the number of 
classes taught ranged from 1 to 6 per teacher. The number of students per teacher ranged from 30 
to 144 each. 

The student data for five ISLE science teachers was comprised of 445 students in 25 
classes. It is important to note that these teachers were directly associated with the Master of Arts 
in Teaching (MAT) program offered through the Science/Mathematics Education Department 
(University of Texas at Dallas) and had completed the summer 2000 field trip based on the ISLE 
model. Approximately six months after the final meeting of the ISLE program, the researcher 
mailed the requested number of surveys to the participating ISLE science teachers for 
independent administration at their discretion. In this phase, the CLES-CS was administered to 
445 students of 5 ISLE science teachers to assess the degree to which the principles of 
constructivism were evident in specific classroom learning environments within the broader 
context of the school-level environment. 

Unable to pre-determine which teachers might actually complete the ISLE program, the 
instrument was trialled with potential candidates who had participated in traditional field trips 
offered by the same instructors in prior years. For comparison purposes, the student data for five 
science teachers, who had participated in alternative field trips programs, was comprised of 328 
students in 19 classes. It is important to note that these teachers also were associated directly 
with the Master of Arts in Teaching (MAT) program through the Science/Mathematics 
Education Department (University of Texas at Dallas), but had completed previous MAT field 
trips that were not based on the ISLE model. Coincidentally, one of the teachers surveyed before 
the ISLE implementation (LH) also participated in the ISLE program. This dual administration 
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not only improved the statistical rigor of the instrument validation (supported by the total sample 
of 1079 students of 10 science teachers), but also provided a representative control group (328 
students of 5 science teachers who had participated in other field trip programs) for comparing 
the effects of the ISLE model. 

The teachers were asked to emphasise that there are no right or wrong answers as the 
students’ opinions were what was wanted. Students were encouraged to think about how well 
each statement describes what the classes are like for them personally, comparing how often 
each practice occurred in THIS particular science class to OTHER classes. Students were 
directed to read each statement and think about lessons they had been taught, indicating the best 
response for the teaching in OTHER classes in the left column. Then they were encouraged to 
read the statement again, and think about lessons that they had been taught, indicating the best 
response for the teaching in THIS class in the right column. 

3. RESULTS 

The student survey responses were recorded in an electronic spreadsheet by the 
researchers. To validate the comparative student form (CLES-CS), the responses of 1079 school 
students were subjected to factor analysis (SPSS for Windows, Release 10.0.5, Standard 
Version) to check the scale structure. Cronbach’s alpha coefficient was used as an index of 
internal reliability and ANOVA (Gay & Airasian, 2000) was used to check whether each scale 
was capable of differentiating between the perceptions of students in different classrooms. To 
compare the students’ perceptions of the classroom learning environment fostered by the ISLE 
teachers to the classroom learning environments fostered by other teachers at their same school, 
the CLES-CS was administered to the ISLE science teachers’ public/private school students (N = 
445). Data were examined using a two-tailed t test. The effect size (Becker, 1999) was also 
calculated using the means and standard deviations to portray the magnitude of differences 
between groups (Rubin & Babbie, 1993). 

3.1. Factor Analysis of the CLES-CS 

Factor analysis (Kim & Mueller, 1982) is a statistical technique used in data reduction to 
identify a small number of underlying variables, or factors, that explain most of the variance 
observed in a much larger number of manifest variables. Using both cases (THIS and OTHER) 
of the CLES-CS data, factor and item analyses were conducted in order to identify faulty items 
that could be removed to improve the internal consistency reliability and factorial validity of the 
five scales in the comparative student version of the CLES. As frequently used in the validation 
of learning environment instruments, the student data were subjected to principal components 
factor analysis with varimax rotation (in which the factor axes are kept at right angles to each 
other) to check the scale structure. 

Four items appeared to be problematic for the students: item 6 was reverse-scored; item 7 
was negatively-worded; and items 3 and 25 were ambiguously interpreted. Removal of items 3 
and 6 in the Personal Relevance scale, item 7 in the Uncertainty of Science scale, and item 25 in 
the Student Negotiation scale enlianced the reliability and factor structure of the instrument. 
Following removal of these four items, all of the other 26 items had a factor loading of at least 
0.4 on their a priori scale and no other scale for the analyses for both THIS and OTHER. Table 3 
presents the resulting factor loadings for both cases of the CLES-CS. 
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Table 3. Factor Loadings for the Constructivist Learning Environment Survey - 

Comparative Student (CLES-CS) Form 

Factor Loading 

Personal Uncertainty Critical Shared Student 

Relevance of Science Voice Control Negotiation 

THIS OTHER THIS OTHER THIS OTHER THIS OTHER THIS OTHER 



1 


0.66 


0.56 


2 


0.48 


0.44 


4 


0.71 


0.63 


5 


0.63 


0.59 



8 0.67 0.63 

9 0.54 0.65 

10 0.44 0.54 

11 0.64 0.68 

12 0.51 0.62 



13 


0.60 


0.56 


14 


0.64 


0.60 


15 


0.65 


0.62 


16 


0.48 


0.46 


17 


0.55 


0.51 


18 


0.53 


0.52 



19 


0.66 


0.61 


20 


0.57 


0.57 


21 


0.75 


0.72 


22 


0.71 


0.73 


23 


0.76 


0.74 


24 


0.54 


0.47 



26 


0.62 


0.58 


27 


0.70 


0.71 


28 


0.79 


0.75 


29 


0.69 


0.66 


30 


0.65 


0.66 



% var- 
iance 


7.0 


5.6 


8.0 


8.1 


8.7 


7.5 


11.5 


10.2 


10.3 


9.2 


Eigen- 

value 


1.83 


1.51 


2.06 


2.18 


2.26 


2.02 


2.98 


2.75 


2.67 


2.49 



N = 1079 students in 59 classes in north Texas. (Items 3, 6, 7, and 25 were omitted.) 

THIS refers to the science teachers’ current class; OTHER refers to classes taught by other non-science 
teachers in the same school. 



Principal components factor analysis with varimax rotation and Kaiser normalization 
confirmed the a priori structure of the CLES-CS. The percentage of the total variance and 
eigenvalue associated with each factor are also shown at the bottom of Table 3. The total amount 
of variance accounted for hy the 26 items within the five scales is 45.5% for THIS and 40.6% for 
OTHER, and ranged from 5.6% to 11.5% for different scales and cases. The eigenvalues range 
firom 1.83 to 2.98 for THIS and from 1.51 to 2.75 for OTHER. Overall, these data provide strong 
support for the factorial validity of the five-scale comparative student version of the 
Constructivist Learning Environment Survey (CLES-CS). 



er|c 



Nix, Fraser, & Ledbetter 



9 



Page 8 



Evaluating an Integrated Science Learning Environment (ISLE) 
using a new form of the Constructivist Learning Environment Survey (CLES) 

3.2. Internal Consistency Reliability and Discriminant Validity of the CLES-CS 

Reliability analysis explores the properties of measurement scales and the items of which 
they are comprised. Cronbach’s alpha coefficient was used as an index of internal consistency 
reliability for each of the scales for two imits of analysis (individual and class mean). Table 4 
shows that the alpha coefficients of different CLES-CS scales were high, ranging from 0.74 to 
0.85 for THIS and from 0.68 to 0.83 for OTHER with the individual as the unit of analysis. 
Using the class mean as the unit of analysis, scale reliability estimates ranged from 0.87 to 0.93 
for THIS and from 0.69 to 0.88 for OTHER. 

To assess the extent to which a scale is unique in the dimension that it covers and is not 
included in another scale in the same instmment, the mean correlation of a scale with other 
scales, also reported in Table 4, was used as a convenient index of discriminant validity. In the 
teachers’ current classes (THIS), the mean correlation of a scale with the other scales varied 
between 0.28 and 0.32 with the individual as the unit of analysis and between 0.28 and 0.39 with 
the class mean as the unit of analysis. In classes taught by other teachers (OTHER), the mean 
correlation of a scale with the other scales varied between 0.25 and 0.27 with the individual as 
the unit of analysis and between 0.16 and 0.34 with the class mean as the unit of analysis. These 
results suggest that each scale assesses a unique dimension and that, while there is some overlap 
between raw scores on scales, they are relatively independent of each other. Additionally, the 
factor analysis results support the independence of factor scores. 

Table 4. Internal Consistency Reliability (Cronbach Alpha Coefficient), Discriminant Validity 

(Mean Correlation with Other Scales), and Ability to Differentiate Between 
Classrooms (ANOVA Results) for Two Units of Analysis for the Constructivist 
Learning Environment Survey - Comparative Student (CLES-CS) Form 





Unit of 
Analysis 




Alpha 


Mean Correlation with 


ANOVA 


Scale 


Reliability 


other Scales 




eta^ 




THIS 


OTHER 


THIS 


OTHER 


THIS 


OTHER 


Personal 


Individual 


0.75 


0.68 


0.29 


0.25 


0.20** 


0.07 


Relevance 


Class Mean 


0.91 


0.69 


0.35 


0.22 






Uncertainty 


Individual 


0.74 


0.78 


0.32 


0.26 


0.18** 


0.11** 


of Science 


Class Mean 


0.87 


0.87 


0.39 


0.18 






Critical 


Individual 


0.77 


0.74 


0.28 


0.26 


0.12** 


0.09** 


Voice 


Class Mean 


0.87 


0.80 


0.35 


0.34 






Shared 


Individual 


0.84 


0.83 


0.28 


0.27 


0.12** 


0.07 


Control 


Class Mean 


0.91 


0.84 


0.28 


0.16 






Student 


Individual 


0.85 


0.82 


0.31 


0.27 


0.12** 


0.10** 


Negotiation 


Class Mean 


0.93 


0.88 


0.38 


0.29 







**p < 0.01 

N = 1079 students in 59 classes in north Texas. (Items 3, 6, 7, and 25 were omitted.) 

THIS refers to the science teachers’ current class; OTHER refers to classes taught by other non-science 
teachers in the same school. 

The eta^ statistic (which is the ratio of ‘between’ to ‘total’ sums of squares) represents the proportion of 
variance explained by class membership. 
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3.3. Ability of the CLES-CS to Differentiate between Classes 

A desirable characteristic of the actual form of a classroom environment scale is that it is 
capable of differentiating between the perceptions of students in different classrooms. Students 
in the same class should see its enviro nm ent relatively similarly, whereas average class 
perceptions should vary from class to class. A one-way analysis of variance (ANOVA) was 
performed for the scores to determine the ability of each CLES-CS scale to differentiate between 
the perceptions of students in different classrooms. Table 4 reports the results in terms of eta^, 
which is the ratio of ‘between’ to ‘total’ sums of squares. 

The eta statistic provides an estimate of the strength of association between class 
membership and the dependent variable (CLES-CS scale). The amount of variance in scores 
accounted for by class membership (eta^) ranged from 0.12 to 0.20 for THIS and from 0.07 to 
0.11 for OTHER in the different CLES-CS scales. The results were statistically significant {p < 
0.01) for nearly all scales and cases, with the exception of the OTHER case for Personal 
Relevance and Shared Control. This suggests that nearly all scales of the CLES-CS are able to 
differentiate between the perceptions of students in different classes. 

3.4. Comparison of Classrooms for ISLE Students (THIS versus OTHER) 

To show the differences between the students’ perceptions of the learning environments 
in the ISLE science teachers’ classroom (THIS) versus the overall environment for other 
teachers’ classroom throughout the same school (OTHER), Figure 2 graphically contrasts the 
average item mean scores of the CLES-CS using the individual as the unit of analysis. Note that 
the maximum range of values varies by a total of 1.52 units. 




SCALE 

* — THIS (N=445) --a-- OTHER 



Figure 2. Students’ Perception of ISLE Science Classroom Teaching: Results of the 

Comparative Student Form of the Constructivist Learning Environment Survey 
(CLES-CS) 
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For the ISLE science teachers’ students, differences between the average item mean 
scores for THIS and OTHER were +0.19 for Personal Relevance, +0.94 for Uncertainty of 
Science, +0.02 for Critical Voice, -0.04 for Shared Control, and -0.02 for Student Negotiation. 
The range of these values (from 2.03 to 3.55) indicates that the practices encompassed by all 
scales of the CLES were perceived by the students to occur with an overall frequency of between 
Seldom (3.00) and Often (4.00) in the public/private schools for both ISLE science and other 
teachers. 

The small differences between scores for THIS and OTHER on three of the scales of 
Critical Voice, Shared Control, and Student Negotiation suggest consistent perceptions about 
administrative policy and classroom management policy. Because this sample was limited to 
science teachers only, data for the other two scales (Personal Relevance and Uncertainty of 
Science) were likely to be skewed in the positive direction due to the specific emphasis on 
science-related content. The average item mean score for Personal Relevance was 3.40 for ISLE 
teachers and 3.21 for teachers who had participated in other field trip programs. These data 
suggest that students perceive science as personally-relevant more often than not. The greatest 
difference in average item mean scores was reported for Uncertainty of Science, ranging from 
3.55 for ISLE teachers to 2.61 for teachers who had participated in other field trip programs. 
These data suggest that the ISLE teachers might present science in a way that demonstrates the 
uncertainty of science more often than teachers who attended alternative field trip programs. 

In order to further investigate the differences in students’ perceptions of the constructivist 
approaches present in their current ISLE science teacher’s class (THIS) as compared with other 
teachers’ classes (OTHER), scores were examined using a two-tailed t test for dependent 
samples. Also, the effect size was calculated using the means and standard deviations of two 
groups (THIS and OTHER) to portray the magnitude of the differences between the groups. 
Table 5 presents the results for each scale, assuming equal variances. 

The data in Table 5 show that differences between the classroom enviro nm ents of THIS 
(ISLE science teachers) and OTHER (other teachers in the same school) are statistically 
significant (p < 0.01) for Personal Relevance and Uncertainty of Science. Not surprisingly, this 
indicates that students perceive the ISLE science classroom as more relevant and uncertain in 
terms of content. Atp> < 0.05, the difference between THIS and OTHER for Shared Control is 
also statistically significant, yet in the opposite direction. These data suggest that students of 
ISLE teachers might not feel as comfortable about opening discussion within the science 
classrooms as do students of teachers who attended other field trip programs. However, this 
unexpected discrepancy could likely be attributed to the nature of the subject. For example, 
literature lessons might be based primarily on group review and interactive dialogue. Science 
lessons are typically based on experimentation that could be perceived as involving the reporting 
of concrete information rather than independently formulated hypotheses. This might impact on 
the students’ perceptions of the learning environment in that their questioning and participation 
are elicited in other classes, while student questioning and participation is inherently enacted in 
the science classroom. 
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Table 5. Average Item Mean, Average Item Standard Deviation, and Differences Between 

ISLE Science Students’ Perceptions of THIS and OTHER Classroom Environments 
with the Individuai as the Unit of Analysis 



Scale 


Items 


Average Item 
Mean 


Average Item Standard 
Deviation 


Difference 






THIS 


OTHER 


THIS 


OTHER 


Effect Size 


t 


Personal 

Relevance 


4 


3.40 


3.21 


0.84 


0.75 


0.12 


4.50** 


Uncertainty 
of Science 


5 


3.55 


2.61 


0.78 


0.95 


0.48 


20.86** 


Critical 

Voice 


6 


3.10 


3.08 


0.90 


0.87 


0.01 


0.63 


Shared 

Control 


6 


2.03 


2.10 


0.83 


0.83 


0.04 


-2.39* 


Student 

Negotiation 


5 


2.84 


2.86 


0.99 


0.93 


0.01 


-0.86 



* p < 0.05, t = 1 .96; ** p < 0.01 , t = 2.58 (Items 3, 6, 7, and 25 were omitted.) 

N = 445 students in 25 classes taught by 5 ISLE science teachers. 

THIS refers to the ISLE science teacher’s current class; OTHER refers to classes taught by other 
teachers in the same school. 



The effect size (i.e. the difference between means expressed in standard deviation units) 
for each scale is also shown in Table 5. The effect sizes range from nearly nothing for Critical 
Voice, Shared Control, and Student Negotiation (0.04 to 0.01), to approximately one tenth of a 
standard deviation for Personal Relevance (0.12) and up to almost half of a standard deviation 
for Uncertainty of Science (0.48). The smaller effect sizes could suggest that the areas over 
which the administrative units appear to have strict control are resistant to change based on the 
ISLE program. By the same token, the magnitude of the larger effect sizes suggests that the ISLE 
program could be having an educationally important effect in improving the learning 
environment indicators over which the teachers evidently feel they have some control. 

Additional qualitative data from the ISLE science teachers’ journals support this 
interpretation. Teacher AC expressed this in the following entry. “Since my curriculum is Life 
Science and very set, the only things that I could incorporate are the medicine plants and some 
information on the plants. I would love to figure out how I could incorporate more”. Teacher RE 
echoed this limitation as well. “Although Erosion and Deposition is now supposed to be in l'’^ 
grade science, I have much more to use when teaching [about the] Cretaceous period, faulting, 
igneous rocks, angle of repose, fossilization, chemistry (and its application), volcanoes, dikes, 
sills, earth history”. 

As an interesting aside, in contrast, qualitative data for the ISLE non-science participants 
suggested creative ways in which they might integrate what they learned into their curricula. 
Teacher GB noted that “I could integrate what I have learned into almost any class, such as, 
English by reading about environments, dinosaurs and ancient history”. The administrator (LL) 
expressed yet another viewpoint in her statement that “I will use this information to view the 
upcoming political issues, on pollution, from a much broader perspective”. 
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3.5. Comparison of the Science Classroom Environments of ISLE Teachers with Teachers 
Who Attended Other Field Trip Programs 

Data for the ISLE science teachers’ classrooms also were compared to results for the 
science classrooms of teachers who attended alternative field trip programs. Using the individual 
as the unit of analysis, Figure 3 graphically presents the average item mean scores for the CLES- 
CS for teachers who experienced ISLE and for teachers who had a different field trip program 
experience (ISLE and non-ISLE). The maximum range of values varies by a total of 1.56 units. 

Students of the science teachers who attended other field trip programs (non-ISLE) 
perceived their science classrooms as slightly more constructivist than did students of the ISLE 
science teachers for two scales (Critical Voice and Student Negotiation). For the science 
teachers’ students, differences between the average item mean scores for ISLE and non-ISLE 
were +0.37 for Personal Relevance, +0.23 for Uncertainty of Science, -0. 1 7 for Critical Voice, 
+0.04 for Shared Control, and -0.05 for Student Negotiation. Again, the range of these values 
(from 2.03 to 3.55) indicates that the practices encompassed by all scales of the CLES were 
perceived by the students to occur with an overall frequency of between Seldom and Often in 
science classrooms in the public/private schools for both ISLE and non-ISLE teachers. 
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Shared Student 
Control Negotiation 

SCALE 

ISLE (N=445) -- 0 -- Non-ISLE (N=328) 



Students’ Perceptions of Science Classroom Environment for ISLE Teachers and 
Teachers from Other Field Trip Programs: Results for the Comparative Student 
Form of the Constructivist Learning Environment Survey (CLES-CS) 



Table 6 shows that differences between the science classroom learning environments of 
ISLE and non-ISLE teachers are statistically significant (p < 0.01) for Personal Relevance and 
Uncertainty of Science. Interestingly, this indicates that students perceive the science classrooms 
of ISLE teachers as more relevant and the topic more uncertain than do students in classrooms of 
teachers who attended other field trip programs. This suggests that the differences in these two 
scales might not be attributable solely to the nature of the course content. In fact, the data 
indicate that Personal Relevance and Uncertainty of Science scales could have been directly 
impacted by the ISLE program. 
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Again, dXp< 0.05, differences between ISLE and non- ISLE teachers are also statistically 
significant for Critical Voice, yet in the opposite direction. This surprising difference suggests 
that students of ISLE teachers might not feel as comfortable about opening discussion within the 
science classrooms as do students of non- ISLE teachers. However, qualitative data suggest that 
this unexpected discrepancy could be a consequence of the overall school-level environment. 
Although the teachers’ experience in the science classroom and the. average number of students 
in each science class did not vary considerably, the basic demographics of the schools did differ 
notably. The non- ISLE science teachers’ schools were all characterised by a large total 
enrolment in urban and suburban settings. In contrast, the ISLE science teachers’ represented 
two small parochial, one medium rural, and only two large suburban schools. This difference in 
overall demographics might account for the students’ different perceptions of the learning 
environment, particularly reflected in the Critical Voice scale. 



Table 6. Average Item Mean, Average Item Standard Deviation, Effect Size, and Differences 

Between ISLE and Other Teachers (Effect Size and f Test for Independent Samples) 
on the Constructivist Learning Environment Survey (CLES-CS) 





Average Item 


Average Item Standard 


Difference Between 


Scale 




Mean 




Deviation 


Programs 




ISLE 


Non-ISLE 


ISLE 


Non-ISLE 


Effect Size 


t 


Personal 

Relevance 


3.40 


3.03 


0.84 


0.96 


0.20 


5.65** 


Uncertainty 
of Science 


3.55 


3.33 


0.78 


0.92 


0.13 


3.68** 


Critical 

Voice 


3.10 


3.27 


0.90 


1.11 


0.08 


-2.28* 


Shared Control 


2.03 


1.99 


0.83 


0.94 


0.02 


0.60 


Student 

Negotiation 


2.84 


2.89 


0.99 


1.07 


0.02 


-0.61 



* p < 0.05, t = 1 .96; ** p < 0.01 , t = 2.58 (Items 3, 6, 7, and 25 were omitted.) 

Nisle = 445 students in 25 classes taught by 5 ISLE science teachers; Nnoh-isle = 328 students in 19 
classes taught by 5 non-ISLE science teachers. 

ISLE refers to science teachers who participated in the ISLE program; non-ISLE refers to science 
teachers who participated in a different field trip program. 



The effect size for each scale is also shown in Table 6. Using the individual as the unit of 
analysis, the effect sizes range from nearly nothing for Shared Control and Student Negotiation 
(0.02), to approximately one tenth of a standard deviation for Uncertainty of Science (0.13), and 
up to one fifth of a standard deviation for Personal Relevance (0.20). As before, the smaller 
effect sizes suggest that the areas over which the administrative units appear to have strict 
control are resistant to change based on the ISLE program. By the same token, the magnitude of 
the somewhat larger effect sizes suggest that the ISLE program could be educationally important 
for improving the learning environment indicators over which the teachers’ evidently feel they 
have some control (i.e.. Personal Relevance and Uncertainty of Science). 
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In summary, the data suggest that the ISLE program was effective in terms of the degree 
of implementation of constructivist teaching approaches in the teachers’ public/private school 
classrooms for the ISLE science teachers, as perceived by their respective students. 

4. Conclusions 

Learning environments research has a broad range of applicability for today’s diverse 
educational issues. Development and validation of the comparative student form of the 
Constructivist Learning Environment Survey (CLES) contributes to a useful range of instruments 
for a variety of classroom contexts within the burgeoning field of learning environments 
research. Our study provided another example of the use of learning environment variables in the 
evaluation of educational programs (Dryden & Fraser, 1998; Maor & Fraser, 1996). The factor 
structure, internal consistency reliability, discriminant validity, and ability to distinguish between 
different classes were supported for the CLES-CS (THIS and OTHER) with our sample of 1079 
students. The overall results validate use of the CLES-CS with students in public/private schools 
in north Texas. 

Participation in the Integrated Science Learning Environment (ISLE) program provided a 
tangible opportunity for teachers to gain organised knowledge to make practical changes in their 
school classrooms. The instructors’ ability to match methodology to outcome offers a broad 
context for enculturation of the constructivist paradigm. Because of the influence of the 
traditional school-level environment, as Milne and Taylor (2000) reported, this sort of 
pedagogical change is difficult to realise in individual classrooms. However, if not appropriately 
implemented, even the best constructivist epistemology is ineffective. The successful 
development of the comparative student form of the Constructivist Learning Environment 
Survey (CLES-CS) provides another, complementary means of evaluating the degree to which 
students feel that the principles of constructivism have been implemented in the class taught by 
their current teacher relative to classes taught by other teachers in their school. 
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Appendix 

Items (Grouped by Scale) on the Comparative Student Form of the 
Constructivist Learning Environment Survey (CLES-CS) 

Item Statement 

Persona! Relevance Scale 

1 I learn about the world outside of school. 

2 My new learning starts with problems about the world outside of school. 

3 I learn how science can be part of my out-of-school life. 

4 I get a better understanding of the world outside of school. 

5 I learn interesting things about the world outside of school. 

6* What I learn has nothing to do with my out-of-school life. 

Uncertainty of Science Scale 

7 I learn that science cannot provide perfect answers to problems. 

8 I learn that science has changed over time. 

9 I learn that science is influenced by people's values and opinions. 

10 I learn about the different sciences used by people in other cultures. 

1 1 I learn that modern science is different from the science of long ago. 

12 I learn that science is about creating theories. 

Critical Voice Scale 

1 3 It's OK for me to ask the teacher ‘why do I have to learn this?’ 

14 It's OK for me to question the way I'm being taught. 

1 5 It's OK for me to complain about teaching activities that are confusing. 

1 6 It's OK for me to complain about anything that prevents me from learning. 

1 7 It's OK for me to express my opinion. 

18 It's OK for me to speak up for my rights. 

Shared Control Scale 

19 I help the teacher to plan what I'm going to learn. 

20 I help the teacher to decide how well I am learning. 

21 I help the teacher to decide which activities are best for me. 

22 I help the teacher to decide how much time I spend on learning activities. 

23 I help the teacher to decide which activities I do. 

24 I help the teacher to assess my learning. 

Student Negotiation Scale 

25 I get the chance to talk to other students. 

26 I talk with other students about how to solve problems. 

27 I explain my understandings to other students. 

28 I ask other students to explain their thoughts. 

29 Other students ask me to explain my ideas. 

30 Other students explain their ideas to me. 

Adapted from Taylor & Fraser (1991) 

* Reverse-scored item 

The response alternatives are Almost Never. Seldom. Sometimes, Often, and Almost Always. 
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